MD5 can be used as a checksum to verify data integrity against unintentional corruption. Historically it was widely used as a cryptographic hash function; however, it has been found to suffer from extensive vulnerabilities. It remains suitable for other non-cryptographic purposes, such as determining the partition for a particular key in a partitioned database. It may be preferred due to lower computational requirements than more recent Secure Hash Algorithms.

History and cryptanalysis

MD5 is one in a series of message digest algorithms designed by Professor Ronald Rivest of MIT (Rivest, 1992). When analytic work indicated that MD5's predecessor MD4 was likely to be insecure, Rivest designed MD5 in 1991 as a secure replacement (Hans Dobbertin did indeed later find weaknesses in MD4).

In 1993, Den Boer and Bosselaers gave an early, although limited, result of finding a "pseudo-collision" of the MD5 compression function: two different initialization vectors that produce an identical digest.

In 1996, Dobbertin announced a collision of the compression functions of MD5 (Dobbertin, 1996). While this was not an attack on the MD5 hash function, it was close enough for cryptographers to recommend switching to a replacement, such as SHA-1 (also compromised since) or RIPEMD-160.

The size of the hash value (128 bits) is small enough to contemplate a birthday attack. MD5CRK was a distributed project started in March 2004 to demonstrate that MD5 is practically insecure by finding a collision using a birthday attack.

MD5CRK ended shortly after 17 August 2004, when collisions for the full MD5 were announced by Xiaoyun Wang, Dengguo Feng, Xuejia Lai, and Hongbo Yu. Their analytical attack was reported to take only one hour on an IBM p690 cluster.

On 1st March 2005, Arjen Lenstra, Xiaoyun Wang, and Benne de Weger demonstrated the construction of two X.509 certificates with different public keys. A few days later, Vlastimil Klima described an improved algorithm, able to construct MD5 collisions in a few hours on a single notebook computer. On 18th March 2006, Klima published an algorithm that could find a collision within one minute on a single notebook computer, using a method he calls tunneling.

Various MD5-related RFC errata have been published. In 2009, the United States Cyber Command used an MD5 hash value of their mission statement as a part of their official emblem.

Applications of MD5 Encryption

Given MD5's cryptographic weaknesses, its primary applications today are in non-security-critical areas where a simple hash function is sufficient. Here are some examples;

Data Integrity Checks

While not as secure as newer hash functions, MD5 can still be used to verify the integrity of small, low-value data. For instance, checking if a downloaded file is complete or corrupted. However, for critical data, stronger hash functions are mandatory.

File Identification

Generating unique identifiers for files based on their content. This can be helpful in file management systems or content delivery networks.

Hash Tables

MD5 can still be used as a hashing function in data structures like hash tables, although other hash functions might offer better performance or distribution.

Deduplication

Identifying duplicate files or data blocks based on their MD5 hashes. This can be useful in data storage and backup systems.

Content-Based Addressing

Creating unique identifiers for content based on its content rather than its filename. This can be used in distributed systems or content delivery networks.

Legacy Systems

In some older systems, MD5 might still be used for various purposes. Upgrading to more secure hash functions is recommended when feasible.

Security

One of the basic requirements of any cryptographic hash function is that it should be computationally infeasible to find two distinct messages that hash to the same value. MD5 fails this requirement catastrophically. 

On 31st December 2008, the CMU Software Engineering Institute concluded that MD5 was essentially "cryptographically broken and unsuitable for further use". The weaknesses of MD5 have been exploited in the field, most infamously by the Flame malware in 2012. As of 2019, MD5 continues to be widely used, despite its well-documented weaknesses and depreciation by security experts.