I believe that, while a high level of scholarship was required to develop the frequency analysis approach, it is not critical to the use of this approach. When the world was new to this subject--when it had just discovered ciphers and keys and cryptanalysis--all of the knowledge was completely new. It was the cutting edge, so not many people understood it yet. It was essential to attain a high level of education to comprehend the mysteries of cryptology. However, with the modern education system, and modern technology, people have the information necessary more readily available. People can access the "mathematics, statistics, and linguistics" necessary to equip themselves for code making and codebreaking. Also, the easy access means that the information surrounds the human population. We have billions of pieces of data sitting at our fingertips, just waiting in that ever-present "cloud." Because of this access, and as a result of the heightened academic expectations, "amateur" cryptanalysts can use previously lengthy and difficult methods of analysis with much more ease. The civilization has reached a "sufficiently sophisticated level of scholarship in several disciplines," and therefore the people of that civilization may achieve the same accomplishments which the Islamic civilization discovered. However, as a result of the constant inundation of information prevalent in our society, and the resultant size of the body of common knowledge, amateur cryptanalysts can now use approaches such as frequency analysis, which was so arduously sought out, without any formal training.
Tag: frequency analysis (Page 1 of 2)
Fundamental development in the disciplinary topics of mathematics, statistics, and linguistics was procured from a comparative ground-level hundreds of years ago as opposed to what we have unearthed today. The advantages and resources currently available to the vast public are, of course, the most they have ever been in history. This goes without saying. "Discovering" tactics to break codes and ciphers that were once considered the most advanced techniques by exceptional cryptanalysts is certainly not as easy a task without the long history of code breaking (in the colloquial meaning of the term) that had come before us.
The ability to learn methods such as frequency analysis from a quick Google search is much less arduous a task than inventing them without any previous notion of such a possibility. Even assuming that today's amateur cryptanalysts aren't explicitly searching "how to's" from public databases, the idea of frequency analysis and any analogous general form of use is very comfortable and familiar. Perhaps teachers from grade school distributed puzzles aimed to unscramble words and phrases or your classmate used a simple cipher as a way to ask out their prom date. Experiencing or seeing a number of similar events throughout our lives inevitably ingrains the technique somewhere in the back of our minds, at least implicitly.
Noting the above, it is truly incredible to acknowledge how commonplace once incredible and cutting-edge discoveries are considered in the present day. This will always be observed, even beyond subjects regarding cryptography, as a natural progression of time.
Each an art form of its own, cryptanalysis and cryptography demonstrate opposing counterparts focused on accomplishing the same common goal—the understanding of a hidden message. These two techniques highlight the competitive battle between codemakers and codebreakers. Although cryptography requires a distinct level of skill and secrecy, the practice of cryptanalysis encounters even greater obstacles as the codebreaker must determine the meaning of the hidden message as well as the technique necessary to break it. Arguably, the mastery of one skill can lead to an expertise in the other as the making of a complex cipher derives the further logic and creativity necessary to uncover these intricate codes.
Singh cites the frequency analysis technique as an “innocuous observation” by Muslim cryptanalysts that became “the first great breakthrough in cryptanalysis” (17). Nowadays, this code-breaking method is quickly and easily used by first-time cryptanalysts with no previous instruction, almost as if by second nature. While the frequency analysis technique was undoubtedly a major breakthrough in the seventh century, the vast amount of education and technology provided to our society today allows this method to become an obvious first step towards discovering the unknown.
As time has gone on and technology has expanded, the human mind has reached a common intelligence almost unimaginable even one hundred short years ago. Education has taught us to not only focus on how to put things together, but also on how to take them apart. Practices such as cryptanalysis have become more applicable to the average man as common knowledge typically requires an understanding of both how and why things work. The fifteenth century Western world is a prime example of the human tendency to discover how something functions as immediately after cryptography was introduced “already there were individuals attempting to destroy this security” (27).
While in the past cryptanalysis was labeled as an expertise only accessible to those in higher society with the finest education, its ability to be understood by even the most amateur cryptanalyst emphasizes the incredible expansion of knowledge in our society today.
The continuous tug-of-war between cryptographers and cryptanalysts has triggered the expansion and prominence of cryptography in the world today. Just as quickly as new encryption methods are developed, new decryption strategies are implemented, creating a state of equilibrium in cryptography.
The art of secret writing dates back to Herodotus (Singh 3) and has had various uses in history, including military communication and murder plots. In more recent years, the explosion of technology has propelled the expansion and necessity of cryptography to the point where it has an effect, either direct or indirect, on every individual. Cryptography has a range of important functions, such as keeping bank account information private and ensuring that government information is kept secret.
Cryptanalysis was not invented until the Islamic golden age under the Abbasid caliphate (Singh 14). According to Singh, cryptanalysis could not be invented until a civilization had reached a sufficiently sophisticated level of scholarship in several disciplines, including mathematics, statistics, and linguistics (15). The Muslim civilization provided an ideal cradle for cryptanalysis because the Muslims were educated in a variety of fields and pursued knowledge in all of its forms (Singh 15).
This breadth of knowledge associated with the Muslim civilization is valued in most modern society today. Most individuals are at least somewhat proficient in an assortment of disciplines. Primary education sets the field by educating children about a variety of different subjects in order to equip them with a large intellectual toolset. Many secondary education institutions stress a liberal education in which students obtain an educational background in the humanities as well as the math and science fields.
Because of this, it is no longer necessary to undergo formal education in cryptanalysis in order to use elementary strategies such as frequency analysis. With the broad education that modern citizens receive, combined with the availability of abundant resources, especially those on the Internet, many of the older, simpler codes can be decrypted by the layperson. However, modern encryptions are much more complex than those of the past and may require serious study.
During the past few centuries, cryptography has exploded due to the expansion of technology. Cryptography affects numerous fields ranging from government information to banking, and brings in knowledge from a variety of areas including mathematics, logic, linguistics, and statistics. Modern education allows us to engage in the world of cryptography even as the complexity and number of cryptographic techniques increase exponentially.
The part of Cryptonomicon that caught my attention was Lawrence Waterhouse's attempt to solve the cipher, or "mathematical exercise" given to him and others by Commander Schoen. Schoen writes out the cipher, a list of 5 groups each with 5 numbers. The numbers are either 1 or 2 digits. Waterhouse instantly recognizes that the greatest number provided is 25, thus he assumes that the numbers must represent the letters in the alphabet. He then decides to run a frequency analysis test using the numbers on the board and realizes the number 18 occurs 6 times. Waterhouse then makes an assumption that the number 18 must be the letter E so he mentally substitutes the letter E into the cipher. Next, Waterhouse observes that the opening 4 numbers are '19 17 17 19'. He then knows that if 19 is a vowel, 17 must be a consonant or vice versa, and since 19 is twice as common in the cipher, he assumes that 19 is a vowel (specifically A). Waterhouse then proceeds by using the context of the message as provided by Commander Schoen earlier. Schoen had said that the message was intended for a naval officer. Using this, Waterhouse was able to guess that the first word was ATTACK. Immediately, he saw the rest of the cipher decrypt before his eyes so he stood up and stated his findings emphatically. The message read: "Attack Pearl Harbor December Seven"
The methods Waterhouse used to solve the cipher reminded me of the discussion we had earlier than year about the use of intuition to solve puzzles. Waterhouse used problem solving techniques and pure logic to decrypt the cipher. By assuming the first word was 'ATTACK' and recognizing that the numbers represented individual letters in the first place, Waterhouse demonstrated the problem solving techniques we have all acquired through basic learning.
In Simon Singh’s The Code Book, he states “Cryptanalysis could not be invented until a civilization has reached a sufficiently sophisticated level of scholarship.” If such a sophisticated level of scholarship was needed to invent cryptanalysis, and cryptanalysts were considered to be of ‘higher’ scholarship, why are amateurs capable of decoding ciphers themselves?
Is the world becoming more intelligent? That I do not know. But it is obvious that knowledge is spreading and learning is occurring more vigorously throughout the world than in the past. As children grow, they are expected to conquer challenges using the mind, not their fists, and grow mentally as they progress through the education system. They are expected to continue their path of learning to higher degrees in high school and college, and expand on their critical thinking abilities. Therefore, society's expectations have furthered amateur cryptanalysts ability to decode messages.
Frequency analysis is a very complex approach to cracking a code. Yet our minds are capable of applying frequency analysis subconsciously when the opportunity presents itself. For example, in the word game, Hangman, our mind subconsciously guesses letters that are frequently seen in words (vowels, specific consonants). Ultimately, people can only apply “on their own” approaches if they have been exposed to higher levels of critical thinking. Education and the accessibility of knowledge provide the world with these abilities to critically think and solve problems. Amateur cryptanalysts, though untrained, can accredit their code breaking skills to the education they’ve received from such an early age.
In Simon Singh’s book, The Code Book, it is expressed that “Cryptanalysis could not be invented until a civilization had reached a sufficiently sophisticated level of scholarship in several disciplines, including mathematics, statistics, and linguistics.” I think this is a very true statement because cryptanalysis involves a level of mathematics and linguistics that wasn’t readily available to many people in early civilizations. The argument that amateur cryptanalysts intuitively use frequency analysis on substitution ciphers and therefore a level of scholarship isn’t needed to solve ciphers is truly invalid in my mind. One needs to realize how different modern civilization is compared to the other time periods we are comparing it to. Along with the civilization needing to reach certain levels of scholarship in mathematics, statistics, and linguistics, they also needed to have to time, energy, and willpower to sit down and try to solve ciphers. In the time of Queen Elizabeth and Queen Mary, the normal folk were illiterate and were working very hard to just have enough food to survive. If they came across a note that looked like a code, they first of all wouldn’t have enough leisure time to try to break the code if they wanted to and they also wouldn’t be literate so they wouldn’t even know where to begin.
In conclusion, the level of education in the modern era is light years ahead of what it was many hundred years ago. One example I heard in class that made it very clear to me was that a hundred years ago, a scholar that was very specialized and on a career path towards engineering or something like that would be the only person who would even considering studying physics. In today’s world, almost every high school graduate has taken a full course in physics.
The level of knowledge required by cryptanalysts to cipher and understand codes has remained relatively the same throughout history. The difference is that this breadth of knowledge is easily developed and acquired in this day in age. Today’s society demands a higher tier of critical thinking in everyday activities due to the advancements in technology and progression of arithmetic and language. An eight year old kid nowadays has the knowledge that a seasoned cryptanalyst had at his disposal long ago. Much like the standard of life, the standard of knowledge has risen exponentially. People involuntarily and unknowingly find themselves in the midst of ciphers and codes every day, therefore peoples’ minds have become conditioned habitually to decode and decipher messages in puzzles and word games.
The knowledge and use of frequency analysis in the alphabet is a more modern practice in cryptanalysis and developed because people are exposed to puzzles, riddles, and word games at an earlier developmental stage. In the end, humans are more frequently immersed in codes and ciphers because of the advancements in thinking and technology. This allows a higher level of cryptanalysis to be performed by ordinary people only due to the fact that they have been subliminally exposed to it their entire lives. People now just have a larger pool of knowledge and experience to pull from, giving them the upper hand in cryptanalysis.
Certainly, the level of scholarship experienced today is far more advanced than that of the past. Each day, each hour, each second advances are made that further expand the breadth of human knowledge; however, this knowledge stands on multiple foundations of understood ideas, preconceived notions, and intuitive reasoning. Cryptanalysis today can be accomplished by those of relatively common education because of the foundations set by previous eras. Thus, as Singh describes, civilizations in the past required more time, effort, and education to establish the principles of frequency analysis simply because they had not acquired the foundation of knowledge that is common today. In the era in which frequency analysis was invented, an "educated person" of the society would have the knowledge to read, write, and do arithmetic, and those skills were a great accomplishment for that era. Yet today eight-year-olds throughout the world have already acquired those skills, along with ways to collaborate with others, communicate effectively, and think critically. This advancement was only possible with a strong foundation of knowledge.
Critical thinking skills also allow ordinary cryptanalysts to intuitively use frequency analysis without training. Today's constantly shifting society forces each person in it to think critically every day. From the rate that technology is advancing to daily medicinal miracles to constant national turmoils, each day provides an opportunity to create something new, something more effective, something no one else could conceive. In this modern society, critical thinking is vital to survival. Conversely, in the years in which frequency analysis was invented, civilizations expanded at a much slower rate, and thus critical thinking at the rate at which it occurs today was much less prevalent in earlier populations.
Today even amateurs, given time, are intuitively predisposed to recognize otherwise “logical” patterns strewn in ciphers and codes alike. This is due to the fact that analytic methods now utilized in the realm of cryptanalysis are not products of innate understanding, but a general improvement of societal and formal education as a whole. Frequency analysis today, for example, proves a fundamental and rather elementary conceptual strategy in decrypting an enciphered message. In the wake of cryptanalysis, however, frequency analysis was a novel and unprecedented notion.
During much of cryptanalysis’ infancy, education was a luxury, largely unattainable to the masses of commoners who preoccupied themselves with self-sustaining labors specialized in practicality rather than subjects of intellect. Such individuals would find themselves entirely dumbfounded by the prospect of solving even the simplest encryptions, provided they harbor any extent of literacy. This approach accounts for the inability of the uneducated to resort to frequency analysis, thus furthering Singh’s argument. On the other end of the societal spectrum, scholars managed to stumble upon the prospect of frequency analysis – but only after a considerable amount of time and inquiry. Singh’s argument again proves sound, for frequency analysis incorporates mathematics, statistics and linguistics, itself being a development in all three fields. Therefore, it seems that “a sophisticated level of scholarship” was indeed necessary to consider frequency analysis as a viable approach for solving substitution ciphers – back then. In a temporal context, perhaps a “sophisticated level of scholarship” is not termed appropriately for the overstatement that it is. Amateur cryptanalysts, along with much of the developed world, have likely received a formal education, in which they have been exposed to the very areas of “expertise” incorporated in frequency analysis. Societal exposure similarly promotes the deciphering technique, the best example of which remains “Wheel of Fortune,” which automatically assumes the most frequently repeating English letters as common knowledge amongst contestants. Moreover, modern society in general places great emphasis on cognitive reasoning from infancy to adulthood, surely fueling the tendency to apply frequency analysis by even the most amateur of cryptanalysts.
In hindsight, the development of frequency analysis was indeed a feat of innovative intellect. Today, however, it seems only a natural inclination to attempt such a logical and practical method of deciphering.